Providing scalable and efficient content delivery is becoming more important as the demand for applications such as streaming media grows at an incredible rate. Content distribution Network (“CDN”) and network service providers advocate using network overlays for providing scalable and robust Internet based applications. Typical overlays are administrator configured, and due to the centralized nature of the overlay construction process, it is not feasible to construct large overlays
Recent application-level overlay networks, such as CAN, eCAN, Chord and PAST, are scalable and self-organizing in nature. Nodes in these networks collectively contribute towards a fault-tolerant and administration-free storage space. The basic functionality these systems provide is a distributed hash table (“DHT”). In these systems, an object is associated with a key. Every node in the system is responsible for storing objects whose keys map to the ID of the node (via hashing). Retrieving an object amounts to routing to a node that is responsible for storing that object. The routing path on these overlay networks is at the application-level rather than the IP level.
While elegant from a theoretical perspective, these systems suffer from two major limitations. First, they rely on application-level routing that largely ignores the characteristics of the underlying physical networks. Because the underlying physical characteristics are not taken into consideration, excessive routing delays typically result. Second, they construct a homogeneous structured overlay network, while in reality, the nodes usually have different constraints such as storage, load, packet forwarding capacities, and network connections.
In addition, the typical overlay networks are constrained. In other words, the number of connections for a node is fixed or limited. Because of the constraint, the ability to accurately model the underlying physical characteristics is limited as well. Further, the earlier auxiliary networks do not handle the dynamic nature of the nature of the underlying network well, for example, when nodes exit and new nodes enter the network.